Multi-scale Feature Fusion Module Research Articles

Environmental Risk Identification and Green Finance Development Based on Multi-scale Fusion Recognition Network

AbstractThis paper aims to enhance the resilience of financial enterprises against environmental risks by leveraging financial data analysis tools. The approach involves designing environmental risk assessment indicators and rating criteria. The study utilizes a convolutional neural network model extended by a multi-scale feature fusion module to analyze environmental risk information in the industry. The proposed model achieves impressive results with accuracy (Acc), precision (P), recall (R), and F1 scores reaching 99.09, 96.31, 95.32, and 95.64, respectively. These metrics outperform those of comparison models. The success of this model is anticipated to pave the way for the transformation of green finance through automated industry-level environmental risk assessment. Furthermore, the method’s adaptability extends beyond environmental risks, offering a scalable solution for identifying and assessing environmental risks in various contexts.

An efficient deep learning model for tomato disease detection

Tomatoes possess significant nutritional and economic value. However, frequent diseases can detrimentally impact their quality and yield. Images of tomato diseases captured amidst intricate backgrounds are susceptible to environmental disturbances, presenting challenges in achieving precise detection and identification outcomes. This study focuses on tomato disease images within intricate settings, particularly emphasizing four prevalent diseases (late blight, gray leaf spot, brown rot, and leaf mold), alongside healthy tomatoes. It addresses challenges such as excessive interference, imprecise lesion localization for small targets, and heightened false-positive and false-negative rates in real-world tomato cultivation settings. To address these challenges, we introduce a novel method for tomato disease detection named TomatoDet. Initially, we devise a feature extraction module integrating Swin-DDETR’s self-attention mechanism to craft a backbone feature extraction network, enhancing the model’s capacity to capture details regarding small target diseases through self-attention. Subsequently, we incorporate the dynamic activation function Meta-ACON within the backbone network to further amplify the network’s ability to depict disease-related features. Finally, we propose an enhanced bidirectional weighted feature pyramid network (IBiFPN) for merging multi-scale features and feeding the feature maps extracted by the backbone network into the multi-scale feature fusion module. This enhancement elevates detection accuracy and effectively mitigates false positives and false negatives arising from overlapping and occluded disease targets within intricate backgrounds. Our approach demonstrates remarkable efficacy, achieving a mean Average Precision (mAP) of 92.3% on a curated dataset, marking an 8.7% point improvement over the baseline method. Additionally, it attains a detection speed of 46.6 frames per second (FPS), adeptly meeting the demands of agricultural scenarios.

Multi-view Self-supervised Learning and Multi-scale Feature Fusion for Automatic Speech Recognition

To address the challenges of the poor representation capability and low data utilization rate of end-to-end speech recognition models in deep learning, this study proposes an end-to-end speech recognition model based on multi-scale feature fusion and multi-view self-supervised learning (MM-ASR). It adopts a multi-task learning paradigm for training. The proposed method emphasizes the importance of inter-layer information within shared encoders, aiming to enhance the model’s characterization capability via the multi-scale feature fusion module. Moreover, we apply multi-view self-supervised learning to effectively exploit data information. Our approach is rigorously evaluated on the Aishell-1 dataset and further validated its effectiveness on the English corpus WSJ. The experimental results demonstrate a noteworthy 4.6%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} reduction in character error rate, indicating significantly improved speech recognition performance . These findings showcase the effectiveness and potential of our proposed MM-ASR model for end-to-end speech recognition tasks.

A novel percussion-based approach for pipeline leakage detection with improved MobileNetV2

Pipelines are susceptible to oil and gas leaks during long-distance transportation due to factors such as damage from external forces and aging. However, existing pipeline leakage detection technologies that rely on physical inspections or sensors installed on pipelines are time-consuming and costly. In this paper, a percussion approach based on improved MobileNetV2 is proposed for pipeline leakage detection. Firstly, the influence of pipe leakage size on vibration characteristics was investigated by theoretical analysis and numerical simulation. Subsequently, experiments were conducted to assess the validity of the proposed method. The sounds produced by hammering the pipe under different damage conditions were recorded using a smartphone. The improved MobileNetV2 model was then used for classifying Mel spectrogram and Mel frequency cepstrum coefficient (MFCC) features extracted from the recorded sound signals. This model incorporates a multi-scale feature fusion module, which allows it to capture features at different scales and enhances its ability to differentiate between damage conditions. Experimental results show that using Mel spectrogram as input for the improved MobileNetV2 achieves a higher accuracy compared to using MFCC, with 100% accuracy for identifying leakage damage and 99.87% for classifying leak size. Compared to other methods, the improved MobileNetV2 exhibits superior classification performance while maintaining the lightweight characteristics of the original MobileNetV2. In conclusion, the improved model demonstrates significant enhancements in classification performance and operational efficiency, making it a promising approach for processing percussive signals.

Multi-Scale Fusion Siamese Network Based on Three-Branch Attention Mechanism for High-Resolution Remote Sensing Image Change Detection

Remote sensing image change detection (CD) is an important means in remote sensing data analysis tasks, which can help us understand the surface changes in high-resolution (HR) remote sensing images. Traditional pixel-based and object-based methods are only suitable for low- and medium-resolution images, and are still challenging for complex texture features and detailed image detail processing in HR images. At present, the method based on deep learning has problems such as inconsistent fusion and difficult model training in the combination of the difference feature information of the deep and shallow layers and the attention mechanism, which leads to errors in the distinction between the changing region and the invariant region, edge detection and small target detection. In order to solve the above problems of inconsistent fusions of feature information aggregation and attention mechanisms, and indistinguishable change areas, we propose a multi-scale feature fusion Siamese network based on attention mechanism (ABMFNet). To tackle the issues of inconsistent fusion and alignment difficulties when integrating multi-scale fusion and attention mechanisms, we introduce the attention-based multi-scale feature fusion module (AMFFM). This module not only addresses insufficient feature fusion and connection between different-scale feature layers, but also enables the model to automatically learn and prioritize important features or regions in the image. Additionally, we design the cross-scale fusion module (CFM) and the difference feature enhancement pyramid structure (DEFPN) to assist the AMFFM module in integrating differential information effectively. These modules bridge the spatial disparity between low-level and high-level features, ensuring efficient connection and fusion of spatial difference information. Furthermore, we enhance the representation and inference speed of the feature pyramid by incorporating a feature enhancement module (FEM) into DEFPN. Finally, the BICD dataset proposed by the laboratory and public datasets LEVIR-CD and BCDD are compared and tested. We use F1 score and MIoU values as evaluation metrics. For AMBMFNet, the F1 scores on the three datasets are 77.69%, 81.57%, and 77.91%, respectively, while the MIoU values are 84.65%, 85.84%, and 84.54%, respectively. The experimental results show that ABMFNet has better effectiveness and robustness.

Why Not Both? An Attention-Guided Transformer with Pixel-Related Deconvolution Network for Face Super-Resolution

Transformer-based encoder-decoder networks for face super-resolution (FSR) have achieved promising success in delivering stunningly clear and detailed facial images by capturing local and global dependencies. However, these methods have certain limitations. Specifically, the deconvolution in upsampling layers neglects the relationship between adjacent pixels, which is crucial in facial structure reconstruction. Additionally, raw feature maps are fed to the transformer blocks directly without mining their potential feature information, resulting in suboptimal face images. To circumvent these problems, we propose an attention-guided transformer with pixel-related deconvolution network for FSR. Firstly, we devise a novel Attention-Guided Transformer Module (AGTM), which is composed of an Attention-Guiding Block (AGB) and a Channel-wise Multi-head Transformer Block (CMTB). AGTM at the top of the encoder-decoder network (AGTM-T) promotes both local facial details and global facial structures, while AGTM at the bottleneck side (AGTM-B) optimizes the encoded features. Secondly, a Pixel-Related Deconvolution (PRD) layer is specially designed to establish direct relationships among adjacent pixels in the upsampling process. Lastly, we develop a Multi-scale Feature Fusion Module (MFFM) to fuse multi-scale features for better network flexibility and reconstruction results. Quantitative and qualitative experimental results on various datasets demonstrate that the proposed method outperforms other state-of-the-art FSR methods.

Fine-Grained Detection Model Based on Attention Mechanism and Multi-Scale Feature Fusion for Cocoon Sorting

Sorting unreelable inferior cocoons during the reeling process is essential for obtaining high-quality silk products. At present, silk reeling enterprises mainly rely on manual sorting, which is inefficient and labor-intensive. Automated sorting based on machine vision and sorting robots is a promising alternative. However, the accuracy and computational complexity of object detection are challenges for the practical application of automatic sorting, especially for small stains of inferior cocoons in images of densely distributed cocoons. To deal with this problem, an efficient fine-grained object detection network based on attention mechanism and multi-scale feature fusion, called AMMF-Net, is proposed for inferior silkworm cocoon recognition. In this model, fine-grained object features are key considerations to improve the detection accuracy. To efficiently extract fine-grained features of silkworm cocoon images, we designed an efficient hybrid feature extraction network (HFE-Net) that combines depth-wise separable convolution and Transformer as the backbone. It captures local and global information to extract fine-grained features of inferior silkworm cocoon images, improving the representation ability of the network. An efficient multi-scale feature fusion module (EMFF) is proposed as the neck of the object detection structure. It improves the typical down-sampling method of multi-scale feature fusion to avoid the loss of key information and achieve better performance. Our method is trained and evaluated on a dataset collected from multiple inferior cocoons. Extensive experiments validated the effectiveness and generalization performance of the HFE-Net network and the EMFF module, and the proposed AMMF-Net achieved the best detection results compared to other popular deep neural networks.

FRD-Net: a full-resolution dilated convolution network for retinal vessel segmentation

Accurate and automated retinal vessel segmentation is essential for performing diagnosis and surgical planning of retinal diseases. However, conventional U-shaped networks often suffer from segmentation errors when dealing with fine and low-contrast blood vessels due to the loss of continuous resolution in the encoding stage and the inability to recover the lost information in the decoding stage. To address this issue, this paper introduces an effective full-resolution retinal vessel segmentation network, namely FRD-Net, which consists of two core components: the backbone network and the multi-scale feature fusion module (MFFM). The backbone network achieves horizontal and vertical expansion through the interaction mechanism of multi-resolution dilated convolutions while preserving the complete image resolution. In the backbone network, the effective application of dilated convolutions with varying dilation rates, coupled with the utilization of dilated residual modules for integrating multi-scale feature maps from adjacent stages, facilitates continuous learning of multi-scale features to enhance high-level contextual information. Moreover, MFFM further enhances segmentation by fusing deeper multi-scale features with the original image, facilitating edge detail recovery for accurate vessel segmentation. In tests on multiple classical datasets,compared to state-of-the-art segmentation algorithms, FRD-Net achieves superior performance and generalization with fewer model parameters.

Rail-STrans: A Rail Surface Defect Segmentation Method Based on Improved Swin Transformer

With the continuous expansion of the transport network, the safe operation of high-speed railway rails has become a crucial issue. Defect detection on the surface of rails is a key part of ensuring the safe operation of trains. Despite the progress of deep learning techniques in defect detection on the rails’ surface, there are still challenges related to various problems, such as small datasets and the varying scales of defects. Based on this, this paper proposes an improved encoder–decoder architecture based on Swin Transformer network, named Rail-STrans, which is specifically designed for intelligent segmentation of high-speed rail surface defects. The problem of a small and black-and-white rail dataset is solved using self-made large and multiple rail surface defect datasets through field shooting, data labelling, and data expansion. In this paper, two Local Perception Modules (LPMs) are added to the encoding network, which helps to obtain local context information and improve the accuracy of detection. Then, the Multiscale Feature Fusion Module (MFFM) is added to the decoding network, which helps to effectively fuse the feature information of defects at different scales in the decoding process and improves the accuracy of defect detection at multiple scales. Meanwhile, the Spatial Detail Extraction Module (SDEM) is added to the decoding network, which helps to retain the spatial detail information in the decoding process and further improves the detection accuracy of small-scale defects. The experimental results show that the mean accuracy of the semantic segmentation of the method proposed in this paper can reach 90.1%, the mean dice coefficient can reach 89.5%, and the segmentation speed can reach 37.83 FPS, which is higher than other networks’ segmentation accuracy. And, at the same time, it can achieve higher efficiency.

MAD-UNet: A Multi-Region UAV Remote Sensing Network for Rural Building Extraction.

For the development of an idyllic rural landscape, an accurate survey of rural buildings is essential. The extraction of rural structures from unmanned aerial vehicle (UAV) remote sensing imagery is prone to errors such as misclassifications, omissions, and subpar edge detailing. This study introduces a multi-scale fusion and detail enhancement network for rural building extraction, termed the Multi-Attention-Detail U-shaped Network (MAD-UNet). Initially, an atrous convolutional pyramid pooling module is integrated between the encoder and decoder to enhance the main network's ability to identify buildings of varying sizes, thereby reducing omissions. Additionally, a Multi-scale Feature Fusion Module (MFFM) is constructed within the decoder, utilizing superficial detail features to refine the layered detail information, which improves the extraction of small-sized structures and their edges. A coordination attention mechanism and deep supervision modules are simultaneously incorporated to minimize misclassifications. MAD-UNet has been tested on a private UAV building dataset and the publicly available Wuhan University (WHU) Building Dataset and benchmarked against models such as U-Net, PSPNet, DeepLabV3+, HRNet, ISANet, and AGSCNet, achieving Intersection over Union (IoU) scores of 77.43% and 91.02%, respectively. The results demonstrate its effectiveness in extracting rural buildings from UAV remote sensing images across different regions.

Self-supervised multi-magnification feature enhancement for segmentation of hepatocellular carcinoma region in pathological images

Accurate segmentation of hepatocellular carcinoma regions is a critical step in surgical evaluation of whole slide pathological images. Recently, emerging multi-magnification learning-based methods have shown promise with evaluation of whole slide images. However, traditional multi-magnification learning segmentation models focus on uniformity and fail to effectively exploit different magnification information. The purpose of this study is to develop a novel multi-magnification learning segmentation model by effectively utilizing diverse magnification information. Therefore, we developed a novel multi-magnification feature self-enhancement network to extract magnification-specific features from multi-magnification images via self-supervised learning without extra supervision. Specifically, the proposed network enhances feature information via a super-resolution module based on self-supervision to transfer the representative information into a segmentation encoder, which takes full advantage of the pyramid storage characteristic of whole slide images. Moreover, a multi-scale feature fusion module based on the attention mechanism is used to fuse pretrained multi-scale features through a gated unit block, which is designed to fuse features from generation task into and segmentation task. We evaluated this method using the hepatocellular carcinoma data set from The Cancer Genome Atlas with cross-validation and achieved a 0.829 Dice similarity coefficient, nearly a 3% improvement over state-of-the-art models used for segmentation. To the best of our knowledge, this is the first study of the application of super-resolution as a self-enhancement network for histopathological image segmentation via self-supervised pretext learning. The code will be available at https://github.com/SH-Diao123/Self-Supervised-Multi-magnification-Segmentation.

SoybeanTracer: An In-Field scene property-based framework for high-throughput soybean canopy coverage extraction and evaluation

Soybean is a crucial plant-based protein and vegetable oil source for the global population and a significant cereal-oil dual-purpose crop. Extracting soybean canopy coverage, subsequently implementing evaluation in a low-cost, high-throughput and accurate manner, is highly valuable for soybean breeding and yield increase. Although the utilization of RGB remote sensing images for soybean analysis offers distinct advantages in terms of its cost effectiveness, there are still issues to be addressed. Specifically, the in-field scene properties of soybean canopy data have rarely been developed and there is a gap in automatic evaluation of soybean canopy coverage. For purpose of tackling these pending issues, this study proposed SoybeanTracer, a novel framework designed to achieve accurate soybean canopy coverage extraction and enable specialized evaluation with low cost and high throughput. The proposed framework comprised two components: Soybean Canopy Segmentation Network (SCSNet) and an agronomic soybean coverage evaluation system. Specifically, SCSNet employed a feature extraction network integrating the homogeneity within plot regions and soybean distribution characteristics. To further address the segmentation challenge of soybean plants with sparse growth posed by genotypes, a soybean multiscale feature fusion module was designed. As a result, SCSNet achieved a mean intersection over union (mIoU) and mean pixel accuracy (mPA) of 89.39% and 94.42%, respectively, on soybean canopy segmentation. Therefore, accurate and low-cost extraction of soybean canopy coverage was achieved through the use of RGB remote sensing data. Furthermore, the soybean canopy coverage evaluation system proposed in this study encompassed five levels, characterized by wider spacing at the two ends and narrower spacing in the middle. This agronomic evaluation system had been effectively implemented on 196 plots. The variance in the pairwise levels of the coverage evaluation results demonstrated the robustness of the evaluation system. SoybeanTracer could indeed help to extract soybean canopy coverage and provide valuable reference for implementing soybean genotype selection, laying a useful foundation for other crops in similar scientific tasks. The source code and trained models were available at https://github.com/KLoAS-Research/SCSNet-SoybeanTracer.

Image Classification Method Based on Multi-Scale Convolutional Neural Network

Traditional convolutional neural networks (CNNs) typically use fixed scale convolutional kernels for feature extraction when processing image classification tasks, while ignoring the multi-scale information present in the image. To overcome this limitation, we propose an algorithm based on multi-scale CNNs, which capture features at different levels by introducing convolutional kernels of different scales into the convolutional layer. In this study, we first designed a multi-scale convolutional layer consisting of multiple convolutional kernels of different scales to extract multi-scale features of the image. To further enhance classification performance, we introduced a multi-scale feature fusion module that can effectively fuse features of different scales and classify them through a fully connected layer. Then we conducted extensive experiments on several commonly used image classification datasets. The experimental results show that this network can not only effectively identify and locate hyperspectral image targets in different scenarios, but also reduce missed detections and false positives during the detection process. The average accuracy of the improved model has been improved, and the recognition accuracy of some small markers affected by external factors such as occlusion and lighting has also been improved. In addition, by comparing the detection effect of a single image, the progressiveness and anti-leakage ability of the improved model are proved. The image classification method based on multi-scale CNNs has broad application prospects in image recognition and feature extraction, and can provide valuable reference and reference for research in related fields.

MUSCLE: multi-view and multi-scale attentional feature fusion for microRNA-disease associations prediction.

MicroRNAs (miRNAs) synergize with various biomolecules in human cells resulting in diverse functions in regulating a wide range of biological processes. Predicting potential disease-associated miRNAs as valuable biomarkers contributes to the treatment of human diseases. However, few previous methods take a holistic perspective and only concentrate on isolated miRNA and disease objects, thereby ignoring that human cells are responsible for multiple relationships. In this work, we first constructed a multi-view graph based on the relationships between miRNAs and various biomolecules, and then utilized graph attention neural network to learn the graph topology features of miRNAs and diseases for each view. Next, we added an attention mechanism again, and developed a multi-scale feature fusion module, aiming to determine the optimal fusion results for the multi-view topology features of miRNAs and diseases. In addition, the prior attribute knowledge of miRNAs and diseases was simultaneously added to achieve better prediction results and solve the cold start problem. Finally, the learned miRNA and disease representations were then concatenated and fed into a multi-layer perceptron for end-to-end training and predicting potential miRNA-disease associations. To assess the efficacy of our model (called MUSCLE), we performed 5- and 10-fold cross-validation (CV), which got average the Area under ROC curves of 0.966${\pm }$0.0102 and 0.973${\pm }$0.0135, respectively, outperforming most current state-of-the-art models. We then examined the impact of crucial parameters on prediction performance and performed ablation experiments on the feature combination and model architecture. Furthermore, the case studies about colon cancer, lung cancer and breast cancer also fully demonstrate the good inductive capability of MUSCLE. Our data and code are free available at a public GitHub repository: https://github.com/zht-code/MUSCLE.git.

MEF-UNet: An end-to-end ultrasound image segmentation algorithm based on multi-scale feature extraction and fusion

Ultrasound image segmentation is a challenging task due to the complexity of lesion types, fuzzy boundaries, and low-contrast images along with the presence of noises and artifacts. To address these issues, we propose an end-to-end multi-scale feature extraction and fusion network (MEF-UNet) for the automatic segmentation of ultrasound images. Specifically, we first design a selective feature extraction encoder, including detail extraction stage and structure extraction stage, to precisely capture the edge details and overall shape features of the lesions. In order to enhance the representation capacity of contextual information, we develop a context information storage module in the skip-connection section, responsible for integrating information from adjacent two-layer feature maps. In addition, we design a multi-scale feature fusion module in the decoder section to merge feature maps with different scales. Experimental results indicate that our MEF-UNet can significantly improve the segmentation results in both quantitative analysis and visual effects.

MRFA-Net: Kidney Segmentation Method Based on Multi-Scale Feature Fusion and Residual Full Attention

For the characterization of the kidney segmentation task, this paper proposes a self-supervised kidney segmentation method based on multi-scale feature fusion and residual full attention, named MRFA-Net. In this study, we introduce the multi-scale feature fusion module to extract multi-scale information of kidneys from abdominal CT slices; additionally, the residual full-attention convolution module is designed to handle the multi-scale information of kidneys by introducing a full-attention mechanism, thus improving the segmentation results of kidneys. The Dice coefficient on the Kits19 dataset reaches 0.972. The experimental results demonstrate that the proposed method achieves good segmentation performance compared to other algorithms, effectively enhancing the accuracy of kidney segmentation.

A feature enhancement FCOS algorithm for dynamic traffic object detection

The development of object detection plays an important role in the realisation of fully autonomous driving, and the feature extraction is the key step for object detection. There has been significant difference and scale variation of object features for different road traffic participants (RTPs), meanwhile traditional Convolutional Neural Networks (CNNs) was difficult to extract object features efficiently for small targets. In order to improve the ability of feature extraction, a RTP object detection method combining dynamic convolution and feature enhancement was proposed. The Fully Convolutional One-Stage (FCOS) object detection algorithm was used as baseline. First, the dynamic convolution module was designed in the backbone network to identify different object features to the maximum extent. Second, a dual attention module was designed to filter object feature information while reducing the amount of computation. Finally, in the detection part, the feature expression ability of shallow network was further enhanced by multi-scale feature fusion module, and the effectiveness of the proposed algorithm was verified using Cityscapes dataset. The experimental result indicated that mAP increased by 2.3% compared with baseline. This study can improve the efficiency of RTP detection and contribute to the industrialisation of intelligent connected vehicles.

A novel difficult-to-segment samples focusing network for oral CBCT image segmentation

Using deep learning technology to segment oral CBCT images for clinical diagnosis and treatment is one of the important research directions in the field of clinical dentistry. However, the blurred contour and the scale difference limit the segmentation accuracy of the crown edge and the root part of the current methods, making these regions become difficult-to-segment samples in the oral CBCT segmentation task. Aiming at the above problems, this work proposed a Difficult-to-Segment Focus Network (DSFNet) for segmenting oral CBCT images. The network utilizes a Feature Capturing Module (FCM) to efficiently capture local and long-range features, enhancing the feature extraction performance. Additionally, a Multi-Scale Feature Fusion Module (MFFM) is employed to merge multiscale feature information. To further improve the loss ratio for difficult-to-segment samples, a hybrid loss function is proposed, combining Focal Loss and Dice Loss. By utilizing the hybrid loss function, DSFNet achieves 91.85% Dice Similarity Coefficient (DSC) and 0.216 mm Average Symmetric Surface Distance (ASSD) performance in oral CBCT segmentation tasks. Experimental results show that the proposed method is superior to current dental CBCT image segmentation techniques and has real-world applicability.

Hyperspectral image change detection based on an improved multi-scale and spectral-wise transformer

ABSTRACT Change detection (CD) has a significant application in the remote sensing field. Because of the popularity of hyperspectral image (HSI) and the application of deep learning methods, hyperspectral image change detection (HSI-CD) techniques have been greatly developed. Among them, convolutional neural network (CNN) has garnered the greatest interest in HSI-CD due to their superior feature learning capabilities. However, current CNN-based algorithms have trouble capturing spectral similarity and long-range dependency owing to their intrinsic structural restrictions. Recently, transformers have been shown to extract global dependency from nature images in an extremely efficient way. But it has some difficulties in handling high-dimensional data, such as HSI. To address these issues, we propose an improved multi-scale and spectral-wise transformer (MS-SWT). The proposed MS-SWT is capable of capturing spectral similarity and long-range dependence between bands to enhance the efficiency of the HSI-CD task. Furthermore, to maximize the utilization of spatial information, we present a multi-scale feature fusion module (MFFM) to extract and fuse different dimensions of spatial features. More importantly, a locality self-attention (LSA) module is employed to alleviate the problem of smoothing the distribution of attention scores due to the large number of spectral embeddings. Moreover, we design a channel self-supervised loss function that can capture intrinsic information from the spectral channels to further strengthen the robustness of model training when the training samples are scarce. Lastly, comprehensive experiments present the high performance of our MS-SWT on four bitemporal HSI datasets and demonstrate the superiority of MS-SWT over state-of-the-art approaches.

MTC-Net: Multi-scale feature fusion network for medical image segmentation

Image segmentation is critical in medical image processing for lesion detection, localisation, and subsequent diagnosis. Currently, computer-aided diagnosis (CAD) has played a significant role in improving diagnostic efficiency and accuracy. The segmentation task is made more difficult by the hazy lesion boundaries and uneven forms. Because standard convolutional neural networks (CNNs) are incapable of capturing global contextual information, adequate segmentation results are impossible to achieve. We propose a multiscale feature fusion network (MTC-Net) in this paper that integrates deep separable convolution and self-attentive modules in the encoder to achieve better local continuity of images and feature maps. In the decoder, a multi-branch multi-scale feature fusion module (MSFB) is utilized to improve the network’s feature extraction capability, and it is integrated with a global cooperative aggregation module (GCAM) to learn more contextual information and adaptively fuse multi-scale features. To develop rich hierarchical representations of irregular forms, the suggested detail enhancement module (DEM) adaptively integrates local characteristics with their global dependencies. To validate the effectiveness of the proposed network, we conducted extensive experiments, evaluated on the public datasets of skin, breast, thyroid and gastrointestinal tract with ISIC2018, BUSI, TN3K and Kvasir-SEG. The comparison with the latest methods also verifies the superiority of our proposed MTC-Net in terms of accuracy. Our code on https://github.com/gih23/MTC-Net.